1. Technical Field
The present invention relates generally to the field of processing two dimensional images, and more specifically, to a real time system to automatically improve the resolution, contrast, and clarity of a stream of two dimensional images using real time deconvolution.
2. Related Art
Two dimensional (2D) images obtained from a variety of optical microscopy modalities often contain blur and noise, which reduce contrast, resolution, and clarity of the image. Such defects can result from any number of causes, including lens imperfections, external noise, etc. In fact, any image captured by a digital camera, which is inherently two dimensional in nature, is subject to such defects. Such images are often processed in order to produce a restoration that is a more faithful representation of the real specimen appearing in the image. Unlike many forms of digital imaging, images acquired with optical microscopy, of virtually all modalities, often have significant degradation due to optical limitations. Although optical microscopy optics themselves are of excellent quality when used within their design conditions, they are frequently pressed to the very limits of resolution, or used with specimens that are not within the design conditions. For example, optical microscopy objectives are designed for specific cover-glass thickness, or require tight tolerances on the index of refraction of all materials (immersion oil, cover-glass, embedding medium, and specimens themselves) in the light path. If these conditions are not met, severe spherical aberration is induced. A common situation is to observe a specimen that is in water and is located on the surface of a slide, rather than immediately beneath the cover-glass. This violates two requirements; first the index of refraction of the water is not matched to the immersion oil. Second, the objectives are corrected for specimens to be immediately beneath the cover glass. Numerous processing techniques for improving the image quality of microscopy images are known, and often require substantial computational resources. (See, Hanser, B., Gustafsson, M., Agard, D., Sedat, J., “Application of Phase Retrieved Pupil Functions in Wide-Field Fluorescence Microscopy”, Proceedings of SPIE, 4621, 40, 2002 and Sieracki, C., Hansen, E., “Pupil functions for aberration correction in 3D microscopy”, Proceedings of SPIE, 2412, 99 (1995).)
In many applications, a stream of images are captured, which when played back show motion or movement occurring within an imaged region. The ability to view the images in a real time playback mode allows the user to better analyze the subject matter captured in the images. Modern digital CCD cameras that are typically used in microscopy can capture mega-pixel images at video rates. Enhancing such a volume of data can require significant computational resources, as each image must be separately processed. Given these constraints, an effective restoration or enhancement of an image sequence cannot easily be accomplished in a real time fashion.
As noted, various approaches are known for processing images. Some of the more effective algorithms utilize a deconvolution process, which requires many computations that cannot be done in real time. One such approach for improving image quality involves the use of blind deconvolution algorithm, which is used routinely for processing three dimensional volumes captured by optical microscopes. “Blind deconvolution” refers to restoring an image when the blurring mechanism is unknown and is determined from the image itself. The blind deconvolution method produces an estimate of the point spread function (PSF). This estimated PSF is a characterization of the optical degradation of the image. For a given set of images, the PSF remains moderately constant, as the imaging conditions are not changing for a specific microscopy specimen and microscope slide/immersion oil configuration. A knowledge of the PSF permits a deconvolution process, where optical degradation is removed from the image by calculation.
Algorithms involving non-blind deconvolution, where the blur function is known a priori, such as that provided with a Wiener filter, are also used in the field of reducing noise and blurred data. (See, Ayers, G., and Dainty, A., “Iterative blind deconvolution method and its applications” Optics Letters, Vol. 13 (No. 7):547-549, July 1988.) However, when using such techniques, users must calibrate the system by measuring the blur, or distortions, associated with each image, which is especially cumbersome and slow in real time image treatment. In addition, a user cannot perform the image treatment and image capturing at the same time.
Accordingly, there are needs for a fast, automatic, system for enhancing a stream of two dimensional images that would allow for real time (or near real time) playback.